Fire detection systems using artificial intelligence

ABSTRACT

A system for automatically detecting fires in select areas, and reacting thereto to put out the fires. A stationary, earth orbit satellite, pilotless drone aircrafts or piloted aircraft contains one or more infrared detectors and optical means for detecting small fires when they first occur in fields and wooded areas, preferably where man made campfires and trash dumping are prohibited. A computer in the satellite, drone or piloted aircraft and/or on the ground receives and analyzes image signals of the earth area or areas being monitored and, upon detecting infrared radiation of varying intensity and variable shape, indicative that a fire has started, generates coded signals which are (a) indicative of the coordinate locations of the fire, (b) the extent of the fire, (c) the shape of the area(s) burning, (d) the direction of movement of the fires (e) the speed(s) of the flame fronts, (f) smoke condition, (g) intensity of the fire, (h) fire ball location(s), etc. In one form, a conventional and/or infrared television camera(s) generates image or video signals which are digitized and recorded in memory and/or on tape or disc. Expert system logic, such as fuzzy logic, is used to prioritize dangerous areas to assist in directing fire fighting. Such information, or select portions thereof, is automatically transmitted to one or more earth bound and/or aircraft contained receivers where fire fighting equipment such as helicopter and aircraft containing drainable water or other fire fighting agents is available for immediate dispatch to the fire zone.

This application is a continuation of application Ser. No. 08/552,810,filed Nov. 3,1995, now U.S. Pat. No. 5,832,187.

BACKGROUND

These inventions relate to the field of fire fighting systems andmethods, and more specifically, to comprehensive fire fighting systemsand methods that automatically optimize fire fighting activities byintegrating image acquisition and analysis, expert systems using fuzzylogic rule-based decision making, satellite positioning and trackingsystems and advanced communication methods.

Fires frequently result in significant disasters that cause loss ofpersonal property and human life. The failure to timely detect fires andoptimally control fire fighting activities causes the unnecessary lossof considerable natural resources. Proper management of fires iscomplicated by the large and remote areas in which the fires oftenoccur, the difficulty in detecting the fires at early stages, andproblems associated with efficient dispatching and tracking of firefighting equipment and crews. The Western United States, for example, isparticularly vulnerable to destruction by fire, due to its wide expansesof open forest areas in mountainous terrains and its frequently drycondition. In addition, weather conditions such as high winds contributeto the rapid spread of fires, resulting in the destruction of largeareas.

Fire fighters have adopted several modern technologies to assist in thecoordination of fire fighting activities. For example, the use of twoway radios enables fire fighters to remain in close communication whilecoordinating fire fighting efforts. Helicopters and aircraft frequentlyassist in the attempt to contain fire damage by dropping water or otherfire fighting agents on portions of the fire. More recently, effortswere reported that use airborne video cameras to monitor areas of a fireand that provide real-time video imaging signals to assist fire fightersin assessing the extent and location of fires. Positioning systems suchas GPS (Global Positioning System) have also been suggested for use indata logging images of fires, fire perimeters, dozer lines, boundaries,etc. See “Real Time Image Analysis And Visualization From Remote VideoFor Fire And Resource Management,” Advanced Imagery. May 1994, at pp.30-32, incorporated herein by reference.

The efforts described above represent important advances in firefighting technology. However, they do not take full advantage of modernexpert computer systems, satellite positioning technology andcommunication methods. Importantly, prior fire fighting systems andmethods fail to quickly detect fires, and to optimize and organize theentire fire fighting effort. The need exists for fire fighting systemsand methods that take advantage of modern computer imaging and globalpositioning technology, coupled with expert system decision logic (e.g.,fuzzy logic), to assist in quickly detecting fires and organizing andoptimizing the overall fire fighting effort.

OBJECTS OF THE INVENTION

It is an object of this invention to provide new and improved firefighting systems and methods that integrate expert system computertechnology, image analysis, modern communication and networkingoperations, and precise global positioning technology to optimize firefighting activities.

It is another object of this invention to provide coordinated firefighting systems and methods that detect and specifically locate firesusing satellite, airborne and fixed-mount reconnaissance of selectedgeographic areas.

It is another object of this invention to provide coordinated firefighting systems and methods using remote control pilotless dronereconnaissance.

It is another object of this invention to provide coordinated firefighting systems and methods using expert systems implemented with fuzzylogic rules.

It is another object of this invention to automatically optimize firefighting activities using fuzzy logic analysis of numerous pertinentfire control factors, such as specific characteristics about the fire,surrounding geography, inhabitants or population near the fire, weather,and the availability and known location of the fire fighting resources.

It is another object of this invention is to optimize fire fightingactivities by continuously tracking and monitoring the location of firefighting resources.

It is another object of this invention is to optimize fire fightingactivities by continuously tracking and monitoring changing fire orweather conditions.

It is another object of this invention to track fire fighting resourcesusing modern locating systems such as Global Positioning System or theGlobal Orbiting Navigational System (GLONASS).

It is another object of this invention assist fire fighters inprioritizing areas in which to concentrate fire fighting resources,including, for example, by considering fire control factors such asweather, terrain, population, property value, and the availability andlocation of known fire fighting resources, and further, to optimize andadjust fire fighting priorities on a real time basis as conditionschange.

It is another object of this invention to consider danger to persons orparticularly valuable properties or resources in prioritizing firefighting decisions.

It is another object of this invention to continually acquire and updatedata defining the fire and the location of fire fighting resources asconditions change to optimize fire fighting activities.

SUMMARY OF INVENTION

The above and other objects are achieved in the present invention, whichprovides totally integrated fire detection systems and methods that useadvanced computer, satellite positioning, and communication systems toquickly analyze a large amount of data to detect a fire and optimize anoverall fire fighting effort.

The integrated fire detection and fighting systems and methods of thepresent invention use earth satellites, piloted and drone aircraft andfixed mount cameras to periodically generate and capture images ofselected geographic regions. The video images are computer analyzed todetect and precisely locate a fire at its earliest stages. Computerimage analysis is used to fully characterize the fire, such asidentifying its extent, intensity, flame fronts and rate of growth, etc.Expert system computers based on fuzzy logic rules analyze all knownvariables important to determining a fire fighting strategy. Forexample, the characteristics of the fire, the characteristics of theregion burning (i.e, its terrain, the existence of natural firebarriers, its combustibility, value, population, etc.), actual andpredicted weather conditions, and the availability, type and location offire fighting resources are all analyzed in accordance with optimizedfuzzy logic rules. The results of the fuzzy logic analysis of allavailable fire control factors are used to optimize fire fightingdecisions, and to update detailed graphics displays.

The integrated fire detection and fighting systems and methods of thepresent invention use advanced image gathering and expert systemanalysis to continuously update and optimize fire fighting decisions asconditions change. Continuously during the fire fighting efforts, imagesof the fire and surrounding terrain are obtained and communicated toimage analysis computers. The image analysis computers evaluate theimage data, and other pertinent characteristics, to maintain a fullyupdated database that characterizes critical aspects of the ongoingfire. Advanced weather gathering tools update weather conditionspertinent to or likely to impact the fire, and communicate that data tothe fire control center. The fire control center uses advanced satellitepositioning and associated communication systems to continuously monitorthe precise location of all deployed and available fire fightingresources. The fire control center includes an expert system that usesfuzzy logic rules to continuously evaluate the updated characteristicsof the fire, weather, and terrain, along with the location and type ofavailable fire fighting resources, to continuously optimize firefighting decisions. A graphics monitor is continuously updated toindicate the location and characteristics of the fire, the location andstatus of all fire fighting resources, the existence of actual orpotential danger areas, and the identity of any high prioritysituations.

The integrated fire detection and fighting systems and methods of thepresent invention use advanced satellite locating systems that trackfire fighting resources to speed deployment and rescue operations.During the fire fighting activities, a fire control center obtains viaGPS or other satellite positioning systems the precise location of allfire fighting resources. An expert system at the fire control centermonitors the actual position and movement of the fire fighting resourcesrelative to prior decisions and the determined characteristics of theactual fire to identify potential or actual danger situations. As eachpotentially dangerous situation is identified, the expert system issuesan alarm. The expert system evaluates the known location and type ofavailable fire fighting resources, and recommends or automaticallyinitiates optimal rescue operations.

Applicant describes below in the Figures and Specification the preferredembodiments of his inventions, as defined in the appended claims. It isapplicant's intention that, unless specifically noted, the words andphrases set forth below and in the claims are to be given theirordinary, accustomed and meaning to those of ordinary skill in thepertinent art. In that regard, in this application, several conceptsfrom different arts are combined in an integrated system, andaccordingly, the individual pertinent arts should be consulted.

Moreover, if the specification or claims recite language as a means orsteps for performing a function, unless otherwise noted, it isapplicant's intention that his inventions be construed to include theuse of any and all structures, materials or acts that are capable ofperforming the recited function, including not only the particularstructure, material or acts shown in the specification, but also anyknown or later-developed structure, material or acts that can performthat function, plus any known or later-developed equivalents thereto.Unless otherwise noted, it is not applicant's intent that any element ofany claim be limited to only the specific structure, material or act forperforming a stated or related function.

For example, generic video and infrared scanners are shown andreferenced throughout the specification. It is intended that anyappropriate imaging system, conventional scanner, special scannner (suchas a laser scanner), camera or optical system can be substituted, aslong as it can generate image data that can ultimately be used by acomputer to detect the presence of a fire. Likewise, radio links areshown throughout the specification as one of the preferred forms of acommunication link. However, any appropriate communication link can besubstituted. Further, while the Global Positioning System (GPS) isdescribed as the preferred locating and tracking system, any existing orlater-developed locating system (such as GLONASS, radar, etc.), whethersatellite based or not, can be substituted. Other examples existthroughout the disclosure, and is not applicant's intention to excludefrom the scope of his invention the use of structures, materials or actsthat are not expressly identified in the specification, but nonethelesscapable of performing expressed functions.

BRIEF DESCRIPTION OF DRAWING

The inventions of this application are better understood in conjunctionwith the following drawings and detailed description of the preferredembodiments. The various hardware and software system elements used tocarry out the invention are illustrated in the form of block diagrams,flow charts, neural network and fuzzy logic algorithms and structures inthe attached drawings.

FIG. 1 provides a general system-level diagram of one form of thepresent invention, illustrating the use of surveillance satellites forlocation and fire monitoring, a fire control center, fire fightingresources and the fire to be detected and fought.

FIG. 1A is an overall system-level diagram similar to FIG. 1, but usinga surveillance craft such as an aircraft, or a pilotless drone, for firesurveillance. As mentioned above, although an aircraft or pilotlessdrone is shown in FIG. 1A, any applicable structure, material or act forconducting aerial surveillance can be substituted, such as tetheredballoons, helicopters, remote controlled rockets, etc.

FIG. 2 illustrates a fire control dispatch system for use in the systemof FIGS. 1 and 1A.

FIG. 3 illustrates a method of partitioning the scanned area into uniquesectors and zones useful for computer analysis and display. Again, it isexpressly noted that any equivalent partitioning method can be used. Forexample, shown in FIG. 16 below is a rectangular partitioning scheme.

FIG. 4 is a partition similar to FIG. 3 illustrating the presence offires in the area scanned.

FIG. 5 illustrates one embodiment of a configuration for the scanning,processing and control equipment employed in the surveillance satellite.Although video and infrared scanners are described, any appropriateimaging device can be substituted.

FIG. 6 illustrates a preferred embodiment for a control computer andcommunication system employed in the fire control headquarters.

FIG. 7 illustrates one embodiment of a neural network of the type usefulin the image analysis processors of the present invention.

FIG. 8 illustrates one embodiment of a neural network processing elementused in implementing the neural network of FIG. 7.

FIG. 9 illustrates an alternative embodiment of a neural network imageprocessor using virtual processing elements.

FIGS. 10A, 10B, 10C and 10D, illustrate representative fuzzy logicmemberships useful in an expert system to analyze fire control factors.Although specific fire control factors are shown, such as distance,combustion factor and wind factor, numerous other factors can be addedor substituted for those shown in FIGS. 10A-10D.

FIG. 11 illustrates representative fuzzy logic inference rules useful inan expert system implementation of the invention. Again, numerous otherinference rules can be added or substituted for those shown in FIG. 11.

FIG. 11A illustrates several additional fuzzy logic inference rules thatinclude a specific parameter reflecting the rate of spread of the fire.

FIG. 12 illustrates a representative danger index matrix useful insetting priorities for specific areas according to their respectivedegrees of danger.

FIG. 13 illustrates a representative value matrix useful in settingpriorities for optimally fighting a spreading fire.

FIG. 14 illustrates a sample calculation for a sector priority vectorusing the value matrix and danger index matrix.

FIG. 15 provides a representative fuzzy logic danger index calculation.

FIG. 16 illustrates an alternate embodiment of the invention usingrectangular fire control areas.

FIG. 17 illustrates the use of the rectangular control areas of FIG. 16for computing relative danger values for areas in the map based oncircular sectors and zones.

FIG. 18 illustrates a representative priority matrix for the respectiveareas of the rectangular fire control grid of FIG. 15.

FIG. 19 illustrates an adjacent node priority matrix useful incalculating fire fighting priorities.

FIG. 20 illustrates one embodiment of an overall fire control processflow for the present invention.

The above Figures are better understood in connection with the followingdetailed description of the preferred embodiments of the inventions.

DETAILED DESCRIPTION

FIG. 1 provides an overview of a first embodiment of the fire detectionand fighting systems and methods of the present invention. A firemonitoring or surveillance satellite 10 scans or otherwise obtainsimages of selected areas of the earth (not shown) using well knownscanning, imaging, video or infrared detection equipment. The scannedareas are typically selected by one or more fire control headquarters 20on the earth. The areas that are scanned or otherwise imaged arepreferably areas of high priority, where the use of fires is generallyprohibited or closely restricted. In its simplest form, the scannedareas include large forests, plains, parks, mountain ranges, or otherareas where fire represents a particularly dangerous situation. In morecomplex forms of the invention, the imaged areas can include urban orresidential areas, although the imaging and fire detection operationsbecome more difficult.

The surveillance satellite 10 communicates with the fire controlheadquarters 20 over conventional radio and other data communicationlinks 70. In a preferred form, the fire control headquarters 20 remotelycontrols a dedicated fire surveillance satellite 10 to continuously scanor image particular areas of the earth for potential fires, representedby the graphic 60 in FIG. 1. In addition, the surveillance satellite 10is controlled to continuously scan or image existing fires 60, and tocommunicate further data to fire control headquarters 20 to use inoptimizing ongoing fire fighting activities, as described in more detailbelow. Any conventional data or control communication link 70 can beused to communicate between the surveillance satellite 10 and the firecontrol headquarters 20. Moreover, the surveillance satellite 10 may beany appropriate satellite imaging system, including shared resourcesatellites, such as the many existing military and weather surveillancesatellites that can be programmed to periodically capture and obtainimages for use by the fire control headquarters 20.

As shown in FIG. 1A, in place of or in addition to the surveillancesatellite 10, other surveillance craft 11 such as piloted aircraft,drone aircraft, or tethered balloons (represented collectively in FIG.1A by the airplane 11) are configured to carry conventional video orinfrared imaging equipment to monitor the selected geographic regions.In this embodiment, as images are generated, an advanced geographicpositioning system, such as the Global Positioning System (GPS) 50, isused to compute and transmit to the fire control headquarters 20 theprecise location of the surveillance craft 11 as images are generated.In that manner, the computer system at the fire control headquarters 20precisely determines and monitors the location(s) of fires as they startand spread or recede. As. in FIG. 1, any conventional communication link70 can be used to transmit control information and data between thesurveillance craft 11 and the fire control headquarters 20. Moreover,while the well known GPS system is preferred for providing thegeophysical tracking, other positioning systems, such as the GlobalOrbiting Navigational System (GLONASS) or radar, can be substituted.

The fire control headquarters 20 uses available advanced image analysistools to evaluate the data from the fire surveillance satellite 10and/or surveillance craft 11. The computer image analysis is carried outto detect the presence of a fire 60, and to determine the accuratelocation of the fire 60 in terms of longitude and latitude coordinates.In a less preferred, but still functional, form of the invention, theimage data can be analyzed off site, for example, at leasedsupercomputing facilities (not shown), and the image analysis resultstransmitted to the fire control headquarters 20. Likewise, thesurveillance satellite 10 and surveillance craft 11 may include therequired image processing tools, at which point only the results of theimage analysis are transmitted to the fire control headquarters 20.

The advanced video and infrared imaging and analysis techniquesapplicable to this aspect of the invention are well known, and as aresult, are not described here. Several applicable remote sensing andrelated computer analysis systems and methods are described in the IEEESpectrum (July 1993), particularly, in the articles beginning at pages20, 28, 33 and 46, and in the article “Real Time Image Analysis andVisualization From Remote Video For Fire and Resource Management,”Advanced Imaging (May 1995), at pp. 30-32, and in U.S. Pat. No.5,445,453, each of which is incorporated herein by reference.

The computing systems and methods at the fire control headquarters 20preferably employ expert systems using fuzzy logic reasoning to analyzethe image data received from the surveillance satellite 10 orsurveillance craft 11, or from an off-site supercomputing facility (notshown), and to derive optimum fire fighting strategies. Those strategiesare based on many factors, including in its preferred form, on (1) thedetermined characteristics of the fire, (2) pre-programmed informationcharacterizing the geographic area in the vicinity of the fire, (3)actual and predicted weather conditions, and (4) the availability,location and type of fire fighting resources.

In general, expert systems using fuzzy logic inference rules are wellknown, as described in the following publications, each of which isincorporated herein by reference: Gottwald, Siegried, Fuzzy Sets andFuzzy Logic: The Foundations of Application—from a Mathematical Point OfView, Vieweg & Sohn, Braunschweig Wiesbaden (1993), ISBN 3-528-05311-9;McNeill, Daniel, Fuzzy Logic, Simon & Schuster, N.Y., (1993), ISBN0-671-73843-7; Marks, Robert J. II, Fuzzy Logic Technology andApplications, IEEE Technology Update Series (1994), ISBN 0-7803-1383-6,IEEE Catalog No. 94CRO101-6; Bosacchi, Bruno and Bezdek, James C,Applications of Fuzzy Logic Technology, Sept. 8-10, 1993, Boston, Mass.,sponsored and published by the SPIE-The International Society forOptical Engineering, SPIE No. 2061, ISBN 0-8194-1326-7.

Preferred fuzzy logic rules applicable to this invention derive optimalfire fighting strategies based not only on real time image analysis andpre-programmed area information, but also on the availability and knownlocation of fire fighting resources such as fire fighting trucks 30,aircraft 40, and associated personnel (not shown). Thus, the firecontrol headquarters 20 also uses known GPS technology to monitor, trackand communicate with all personnel and equipment 30,40 available tofight a fire. Various configurations of GPS-based tracking andcommunication systems and methods are described in the followingdocuments, each of which is incorporated herein by reference: Logsdon,Tom, The Navstar Global Positioning System, Van Nostrand Reinhold, NewYork, (1992), ISBN 0-422-01040-0; Leick, Alfred, GPS SatelliteSurveying, John Wiley & Sons, New York (1990), ISBN 0-471-81990-5; Hurn,Jeff, GPS—A Guide to the Next Utility, Trimble Navigation, Ltd.,Sunnyvale, Calif. (1989); Hurn, Jeff, Differential GPS Explained,Trimble Navigation, Ltd., Sunnyvale, CA (1993); and U.S. Pat. Nos:5,434,787; 5,430,656; 5,422,816; 5,422,813; 5,414,432; 5,408,238;5,396,540; 5,390,125; 5,389,934; 5,382,958; 5,379,224; 5,359,332;5,418,537; 5,345,244; 5,323,322; 5,243,652; 5,225,842; 5,223,844;5,202,829; 5,187,805; and 5,182,566. In less preferred, but stillapplicable, forms of the invention, conventional radar or otherpositioning systems and methods can be substituted to locate and trackthe fire fighting resources 30,40.

Thus, in addition to receiving the image data from the satellite 10 andsurveillance craft 11, the computer system at the fire controlheadquarters 20 also receives the GPS location data from each of thefire fighting resources 30,40. In a preferred mode, the controlheadquarters 20 automatically receives the GPS data from each of thefire fighting resources 30,40 over regular, programmed periods of time.Alternatively, the control headquarters 20 periodically polls each firefighting resource 30,40 over communication links 90, and instruct thoseresources to transmit their respective GPS data. In either manner,current and precise location data is obtained for all available firefighting resources 30,40, and is used in the fuzzy logic expert systemsat the fire control headquarters 20 to assist in optimizing the firefighting efforts. Conventional radio, data and control communicationlinks 90 exist between the fire control headquarters 20 and the firefighting resources 30,40.

When the location of all fire fighting resources 30,40 is tracked asdescribed, then the fire control headquarters 20 can optimize firefighting activities by: (a) automatically computer analyzing thecharacteristics of the fire and the area burning, along with the knownlocation of all fire fighting resources; (b) automatically andefficiently directing the closest or most effective fire fightingresource to specific high priority locations in or beyond the fire zone;and (c) immediately and precisely dispatching rescue operations if anyfire fighting resource indicates or is determined to be in danger. Inthe latter example, known GPS-based emergency locating systems can beused to automatically transmit the location of a fire fighting resourceexperiencing an emergency condition. See U.S. Pat. Nos. 5,418,537,5,392,052, 5,367,306 and 5,355,140, each of which is incorporated hereinby reference.

The computer system at the fire control center 20 includes a graphicdisplay monitor that displays a continuously updated map indicating thelocation and condition of the fire, the terrain burning, the locationand type of each fire fighting resource 30,40, the location of eachmobile or remote center 21-26 (see FIG. 2), and the location of airbornetracking craft 11. The image data from the satellite(s) 10 andsurveillance craft 11, as well as the GPS data from each fire fightingresource 30,40, is continuously or periodically communicated to the firecontrol headquarters 20, and is employed to construct and update thedisplayed map of the fire fighting activities. Field or remote commandunits 21-26 also include a graphic display and receiver thatcommunicates with the fire control center 20. In that manner, the fieldcommand units 21-26 also continuously or periodically receive anddisplay graphics showing the current state of the fire and location ofall fire fighting resources 30,40. Illustrative GPS-based mappingsystems that can be used in conjunction with this aspect of theinvention include U.S. Pat. Nos. 5,420,795, 5,381,338, 5,396,254,5228,854 and 5,214,757, each of which is incorporated herein byreference.

In addition to the scanning or imaging equipment carried by thesurveillance satellite 10 and craft 11, selected of the fire fightingresources 30,40 are also configured to carry video or infrared sensorsand associated computers. The image data acquired by the fire fightingresources 30,40 is tagged with time and GPS location data, and is eitherstored for immediate analysis or transmitted via communication link 90to remote (e.g. 21-26) or central 20 fire control headquarters. Thus,the fire control headquarters 20 also receives from selected of the firefighting resources 30,40 either image data or the results of remoteimage analysis operations.

Although FIGS. 1 and 1A graphically depict the fire fighting resourcesas trucks 30 and helicopters 40, it is specifically noted that suchresources include remote controlled fire fighting systems, such aspilotless drones, driverless vehicles or robotic vehicles are alsoemployed in fighting the fires. Combining GPS location data, expertsystem analysis and advanced communication networks allows the central(20) and remote (21-26) fire control centers to automatically andefficiently optimize fire fighting decisions, and if desired, toremotely control fire fighting resources 30,40.

Thus, at least some of the pilotless or driverless craft 30,40 areconfigured to carry and operate special fire fighting equipment, such aswater streaming or dropping equipment, chemical fire fighting dispersalequipment, earth moving equipment, and the like. Machine vision withintelligent and expert computer systems using fuzzy logic and neuralnetworks are employed not only to capture and transmit image data foranalysis by the fire control centers 20-26, but also to control suchremote controlled and robotic resources 30,40. Available GPS-basedremote control guidance systems are used to precisely guide such remotecontrolled and robotic resources 30,40 to desired geographic coordinatesto conduct specified fire fighting activities. Applicable GPS-basedguidance systems include those shown in the following U.S. Pat. Nos.:5,214,757; 5,193,064; 5,220,876; 5,247,440; 5,260,709; 5,270,936;5,334,987; 5,361,212; 5,412,573; 5,420,795; and 5,438,817, each of whichis incorporated herein by reference. Alternatively, other conventionalremote guidance systems can be substituted, such as those employingradar, attached wire, image analysis, inertial guidance, etc., of thetype commonly employed in guiding military missiles to targets.

In operation, remote controlled and robotic resources 30,40 as shown inFIG. 1A are preloaded with water and/or chemicals in the fuselages,wings and/or auxiliary tanks, and placed on standby status. Promptlyupon detecting and locating a fire, the fire control headquarters 20issues via satellite or radio communication links 90 control signals tolaunch and guide the resources 30,40 using applicable guidance systemssuch as those disclosed in the above-referenced and incorporatedpatents. When the mission is completed (for example, due to expenditureof fire extinguishing liquid), the remote controlled resources areautomatically returned to their launching sites or bases, where its fireextinguishing loads are replenished and the craft readied for furtheroperations.

It is also preferred to configure at least one mobile fire controlcenter (indicated in FIG. 2 by the airborne remote control center 21) tocarry backup computer and communication systems similar to thosemaintained at the central fire control headquarters 20. In that manner,even remote or large fires can be effectively monitored and controlledfrom close-in ground positions, or from the air, or from localairfields.

In areas of extremely high risk or frequent fire activities, video andinfrared sensing units and associated image analysis systems aresupported on towers or tethered balloons (referred to below as “localfixed surveillance systems”), and are linked to the fire controlheadquarters 20 or remote fire control. Centers 21-26. An imagingcontrol subsystem within the local fixed surveillance systems monitorhigh risk occurrences, such as lighting strikes, and focuses the imagingsystem in the region of the occurrence to search for an anticipated fire60. Similar “sense and focus” techniques may be incorporated into theimaging systems carried by the satellite 10, surveillance craft 11, andmanned or robotic fire fighting resources 30,40.

Shown in FIG. 2 is a more detailed diagram of a preferred configurationfor the fire control and dispatch system of FIGS. 1 an 1A. As shown inFIG. 2, the fire control headquarters 20 communicates over links 28 witha number of remote or local fire control centers 21-26. The remote orlocal fire control centers 21-26 comprise either or both ground orairborne centers, including mobile centers, which are dispersed atdifferent locations of the area under surveillance. Conventional data,voice and control communication links 28 allow full coordination betweenthe remote control centers 21-26 and to fire control headquarters 20.

The remote control centers 21-26 are available to assist the firecontrol headquarters 30 with interrogating, dispatching, controlling andmonitoring the varied manned and unmanned fire fighting resources 30,40of FIGS. 1 and 1A. In addition, the remote control centers 21-26 containlocal sensing systems, such as surveillance and weather sensors, tocapture, analyze and transmit pertinent information over communicationlinks 28 to each other, to the fire control headquarters, and to thefire fighting resources 30,40. The local weather and fire conditions arecontinuously updated at the remote control centers 21-26, andtransmitted to the central fire control headquarters 20 for use by theexpert system in carrying out automatic fire control computations andderiving optimal control strategies. As shown in FIG. 2, and asdiscussed earlier, precise geographic locating information, such as thatgenerated from GPS satellites, receivers and data transmitters, is alsomonitored at each remote control center 21-26. Each remote controlcenter 21-26 includes a computer and associated graphics display systemthat receives from fire control headquarters 20 an updated fire map andstatus of the pertinent fire fighting activities, as discussed ingreater detail below.

FIG. 3 illustrates a preferred method of partitioning via an electronicdisplay the monitored area in the vicinity of the fire. Partitioning themonitored areas into sectors S1-S8 and zones Z1-Z5 allows precisedetermination, location and designation of danger areas, and aids in theoptimum dispatch of fire fighting resources 30,40. More specifically, inthe method illustrated in FIG. 3, the overall monitored area is dividedinto pie-slice sectors S1-S8, and concentric circular zones Z1-Z5.Specific areas in the partitioned space are identified as areas A_(ij),where i refers to the ith sector and j to the jth zone of the overallarea being scanned. The purpose of such a partitioning scheme is toprecisely locate the indications of individual fires within the entirearea, and further, to precisely indicate high priority areas for theaccurate dispatch of fire fighting resources. If desired, and as thefire fighting activities progress, the overall area being monitored maybe scrolled or scaled, for example, by controlling the imagingoperations of the surveillance satellite 10, aircraft 11, and otherimage acquisition and analysis tools, so that the fire is displayedrelatively centered to scale in the middle of the grid. In addition, thearea A_(ij) may be defined as large or as small as desired.

FIG. 4 illustrates an electronic display defining a fire map using thesector and zone partitioning scheme of FIG. 3, and further showing twofires F3 and F2. As shown in FIG. 4, the fires F3 and F2 are separateand distinct from each other, and their overall contours and relativepositions are electronically and individually indicated on the fire map.The individual sub-areas A_(ij), where fires are present, are easily andspecifically identified. The circular area partitioning illustrated inFIGS. 3 and 4 is useful in visualizing the extent and rate of growth offires in individual sectors. Of course, other area partitioning schemesmay be used, such as rectangular grids, as discussed below.

FIG. 5 is a block diagram of a representative imaging and control system100 for fire surveillance satellite 10 shown in FIG. 1 (or surveillancecraft 11 of FIG. 1A). The imaging and control system 100 of FIG. 5includes multiple video 102 and infrared 103 scanning devices thatcapture images for processing and transmission to the fire controlheadquarters 20. The satellite system 100 includes a control computerand signal routing system 101, which is used to interface the variousvideo and control signals with radio, scanning, and processingequipment. The control processor and signal routing system 101 receivespower from a satellite power supply system 104 and timing from a clocksystem 105. The power supply 104 may be supplied by associated solarpanels (not shown). The clock system 105 is preferably synchronized withknown internationally transmitted timing signals, enabling preciseidentification of date and time information for association withindividual video and/or infrared scanning information derived by thescanners 102 and 103.

While FIG. 1 illustrates the use of a fire monitoring satellite 10, itwill be apparent to those skilled in the art that monitoring may beaccomplished from alternative airborne vehicles, such as airplanes,helicopters, pilotless drone, remotely controlled airborne vehicles,balloons, etc. To further illustrate applicant's intent that all suchimage acquisition and surveillance craft are contemplated, FIG. 1Aexpressly substitutes the use of an aircraft or pilotless drone 11 forsurveillance and gathering of necessary data. As shown in FIG. 1A,satellite position indicating systems 50 are used to precisely locateand/or navigate the aircraft or pilotless drone 11. A pilotless craft 11may be controlled and directed from any of the fire control centers20-26 so that they travel to specific locations and scan particularareas for the existence of a fire 60.

As further illustrated in FIG. 5, the surveillance satellite 10 includesseveral conventional image acquisition or video scanners 102,103 thatscan and derive video and infrared signals of select areas on the earth.The video 102 and infrared 103 scanners are preferably individuallycontrolled by control line 112 from the control processor and signalrouting system 101. Such control is in turn preferably derived frominformation code signals received via command control radio links 70(FIG. 1, 1A) or 110, 111 (FIG. 5) from earth, and ideally, from the firecontrol headquarters 20. In that manner, the operation of the videoscanning equipment 102, 103 of FIG. 5 is precisely and continuouslycontrolled to scan and acquire images data from particular and specifiedareas of the earth. Further, such a configuration provides theflexibility necessary to periodically monitor a large number of areas,while more frequently monitoring high priority areas.

The video 102 and infrared 103 scanners are coupled through respectiveanalog to digital (A/D) converters 102A and 103A illustrated in FIG. 5for digitizing image signals and communicating them to the controlprocessor and signal routing circuitry 101. The signal routing circuity101 in turn transmits the data in real time to the fire controlheadquarters 20, or routes the image data to the appropriate memory106,109 and processing circuitry 107,108, as shown in FIG. 5. Thus, thesystem has the ability to acquire, process and analyze image data on thesurveillance satellite 10, or to communicate the image data to the firecontrol center 20 for processing and analysis.

As shown in FIG. 5, the computer-controlled image processor 108,performs preliminary image processing and analysis operations withrespect to image data derived from one or more of the video 102 orinfrared 103 scanners. The image processor 108 is of conventionaldesign, and typically includes a compact, high speed, parallel processorbased, for example, on modern VLSI implementation of neural networkelements or system(s). The image processor 108 may also be implementedusing more conventional parallel processing structures implemented incompact VLSI form suitable for operation in a satellite configuration.The purpose of the image processor 108 of FIG. 5 is to process andanalyze the image data to detect the presence of a fire, and tocommunicate the processed image signals to the fire control center 20 ofFIG. 1 for storage or further evaluation. As illustrated in FIG. 5, theimage processor 108 operates in conjunction with image signal memory 109for the computerized analysis of digitized image signals derived fromthe video scanners and infrared scanners 103. As previously discussed,instead of or in addition to processing the data on board the satellite10 or craft 11, the image data may be communicated directly to the firecontrol headquarters 20 for image processing and analysis. In the lattercase, the image processor 108 and image memory 109 are also resident atthe fire control headquarters 20.

Also illustrated in FIG. 5 is central processor 107, operating inconjunction with a control memory 106, for effecting the automaticcontrol of the various subsystems of the satellite 10. The centralprocessor 107, for example, is in communication with the ground basedfire control headquarters 20 via the radio link 70 as illustrated inFIG. 1. Radio signals are transmitted via the radio transceiver 110 andantenna system 111 of FIG. 5. The central processor 107 operates inresponse to command and control signals stored in control memory 106, incombination with command and control signals received via radio link 70,to control the overall operation of the video 102 and infrared 103scanners, image processor 108, and the transfer of coded or digitizedimage information and command control signals from the satellite 10 tofire control headquarters 20.

FIG. 6 illustrates a preferred embodiment of the processing, control andcommunications systems used at fire control headquarters 20 of FIG. 1.The fire control headquarters 20 receives coded image signals from thecontrol system 100 of the surveillance satellite 10. Communication ofcontrol and data signals between the satellite 10 and fire controlheadquarters 20 occurs via the satellite radio link 70, or any otherappropriate communication link. Typically, data and control signals arereceived at the satellite receiving dish 201, and are transferred from aradio receiver 202 to video preprocessor 203. The video signalpreprocessor 203 decodes video signals (e.g., individual picture frames)for transmission to a video image memory bank 204 via an interconnectingbus mechanism 207. Also received via the satellite link 70 from thesatellite 10, are command decision signals indicative of the presence orabsence of fire, as well as other status and control information. Asdiscussed above, in the configuration of FIG. 1A, the signals arereceived in a similar manner at the fire control headquarters 20 fromother surveillance craft 11, and from various of the remote controlcenters 21-26 and fire fighting resources 30,40.

The video picture signals captured in image memories 204 of FIG. 6 areprocessed in an image processing block 205. The image processor 205 maybe one or more of the numerous available high speed parallel processingcomputers designed to efficiently execute specific image processingalgorithms to derive information concerning a desired phenomenon, inthis case, a fire. The computer is programmed in accordance with commonprogramming techniques to identify a fire, and to characterize thevarious features of the fire, such as the geographic boundaries of thefire, its direction and speed of movement, its intensity, etc. Multiplefires may also be detected using the image processor 205. Specialpurpose parallel co-processors 206 may be used, in a manner well knownto those skilled in the image processing art, for efficient signalprocessing of the data derived via satellite link 70 and the variousother communication links. Such parallel or coprocessing techniques areparticularly useful in implementing certain repetitive and structuredmathematical operations associated with computer controlled imageprocessing, such as high speed matrix manipulation. As generally shownin FIG. 6, the co-processors 206 communicate with the image processors205 and image memories 204 via the communication bus 207. However, theuse of massively parallel co-processors for high speed and detailedimage analysis are well known to those of ordinary skill in the imageprocessing art, as described in the following publications, each ofwhich is incorporated herein by reference: Carpenter, G. A. andGrossberg, S., Neural Networks for Vision and Image Processing, MITPress, Cambridge, Mass., 1992; Kittler, J. and Duff, M., ImageProcessing system Architectures, Research Studies Press LTD.,Letachworth, England, 1985; Pearson, D., Image Processing, McGraw-HillBook Company, New York, 1991; Teuber, J., Digital Image Processing,Prentice Hall, New York, 1993.

The control processor 208 and its associated data and program storagememory 217 control the overall operation of the fire controlheadquarters 20, including the processing and analysis of individualimage signals, derivation of optimal control strategies, communicationwith remote control centers 21-26, surveillance satellites 10 and craft11, and the various fire fighting resources 30,40. As discussed ingreater detail below, the control processor 208 is preferably an expertsystem computing unit operating based on fuzzy logic reasoning or otherforms of artificial intelligence to evaluate all pertinent data on anongoing basis as it is received from the various sources. The controlprocessor 208 is programmed to derive and indicate preferred strategiesfor optimizing fire fighting activities. Expert systems are well knownto those of ordinary skill in the art, as reflected in the followingpublications, each of which is incorporated by reference herein: Harmon,Paul and King, David, Artificial Intelligence in Business—ExpertSystems, John Wiley & Sons, New York (1985), ISBN 0-471-8155-43;Gottinger, H. and Weimann, H., Artificial Intelligence—a tool forindustry and management, Ellis Horwood, New York (1990), ISBN0-13-48372-9; Mirzai, A. R., Artificial Intelligence—Concepts andapplications in engineering, Chapman and Hall, New York (1990), ISBN0-412-37900-7; Bourbakis, N, Artificial Intelligence Methods andApplications, World Scientific, New Jersey (1992), ISBN 981-02-1057-4;Schalkoff, R., Artificial Intelligence: An Engineering Approach,McGraw-Hill, New York (1990), ISBN 0-07-055084-0; Frenzel Jr., L., CrashCourse in Artificial Intelligence and Expert Systems, Howard W. Sams &Co., Indianapolis, Ind., (1987), ISBN 0-672-22443-7.

A preferred expert system implementation based on a fuzzy logic approachto fire fighting and control is further described below. The controlprocessor 208 makes use of information received from the databasecomputer 209. The database computer 209 maintains and draws informationfrom the database storage unit 210. The database storage unit 210contains extensive information describing the condition of terrainthroughout the region(s) being monitored by the overall fire detectionand control system, and continues to store and learn from new datareceived from the numerous sources discussed above. Terrain data forindividual areas throughout the monitored region is maintained in thedatabase unit 210, and periodically updated by well known GPS-basedlogging methods. Also included in the data base unit 210 is informationreflecting the relative value of properties located in the monitoredregions. Even after a fire is detected, the terrain and relative valueinformation is updated on a continuous basis using, for example,information from the remote control centers illustrated in FIG. 2 anddescribed above. Real-time information relating to the physicalcondition of the terrain and indicative of the degree of fire hazard inthe area(s) under surveillance as influenced, for example, by theexistence of drought or particularly dense combustible materials, isrecorded in the memory of the database 210. This information is usedtogether with the results of the analysis of image data received fromthe satellite 10 and surveillance craft 11, as described above, toderive the optimal fire fighting control strategies.

Code signals defining the results of computer processing and analysis ofthe various fire control factors (e.g., terrain, weather, etc.) arerouted to an electronic display and input/output (I/O) processor 211.The display processor controls various types of display monitors andterminals 212 to show the current status of all tracked information. Forexample, at least one of the display terminals includes a display of thefire control grid of FIGS. 3 and 4, which includes a display of the fireand graphic symbols showing the tracked location of all dispatched andstand-by fire fighting resources. In more complex versions, weather. andother geographic characteristics can be superimposed on fire controlgrid shown at the display terminals 212. The same or other monitors ordisplays 212 are configured to show in a graphic form actual orpotential danger or priority zones. The display processor 211 alsocommunicates over bus 207 with the communication processor 215, to allowtransmission of the composite displays of the pertinent fire controlconditions to displays at remote control centers 21-26 and the variousfire control resources 30,40.

The selected code signals defining the results of computer processingand analysis of the various fire control factors are routed to asynthetic speech generating computer 213 and associated speakers 214,which generate audible speech defining fire control warnings andcommands. For example, if selected alarm condition is detected by thecontrol processor 208, command warnings are both displayed on theterminals 212 and audibly over speakers 214. In that manner, the systemimmediately alerts fire control coordinators of dangerous firesituations and attracts their attention without undue delay which wouldotherwise be possible if only printed reports or visual displays wereused. The speech synthesizer 213 is used not only to gain the attentionof monitor station personnel, such as fire fighting coordinators, butalso to specifically identify to them the areas where fires have beendetected by the described electronic system and to recommend immediateaction, such as the dispatching of nearby or available fire fightingresources 30,40 and to indicate the location(s) of persons present inthe dangerous area(s) so that they may evacuate or be rescued.

Also shown in FIG. 6 is a communication processor 215 connected to aplurality of communication channels or links 216. The communicationprocessor 215 transmits and receives coded command control signals anddata to and from remote control centers 21-26, and communicates firecontrol commands and information directly to the fire fighting resources30,40. The communication network defined by the links 216 may bededicated radio or landline links communicating with remote controlcenters, or may be defined by a communication system comprising anestablished telephone switching system or cellular telephone system tocommunicate operable fire data and command messages to personnel in thefield and to order the dispatch of suitable fire fighting equipment toselect locations. High speed data communication links available viapublic, cellular and radio phone networks may also be used to transmitdigital video picture signals as well as specific command and controlmessages to one or more field sites to graphically depict the fire andindicate dangerous situations to fire fighting personnel. The specifictype of communication system and link is not critical to the invention,and any of the numerous available commercial or military communicationsystems and methods can be substituted.

Shown in FIG. 7 is one embodiment of a neural computing network havingprocessing elements suitable for performing successive computations onimage and other data (e.g., weather). Such neural computing networks areused to carry out the image processing in the computers 108 (of FIG. 5)and 205 (of FIG. 6). The neural network of FIG. 7 includes multipleprocessing elements 130 configured in layered structures. The processingelements (PE's) 130A, 130B and 130C map input signals vectors to theoutput decision layer, performing such tasks as image recognition andimage parameter analysis. Although the layered structure of FIG. 7 isshown as a preferred embodiment, it is noted that any appropriatecomputer processing configuration can be substituted.

A typical neural network processing element or circuit is shown in FIG.8. Input vectors 122 (identified as Xl, X2 . . . Xn) are connected viaweighting elements 132 (identified as W1, W2 . . . Wn) to a summing node150. The output of node 150 is passed through a non-linear processingelement 160 to produce an output signal U. Offset or bias inputs can beadded to the inputs through a weighting circuit 140 (identified as Wo).The non-linear function 160 is preferably a continuous, differentiablefunction, such as a sigmoid, which is typically used in neural networkprocessing element nodes.

In accordance with standard expert system and neural network programmingtechniques, the neural networks used in the fire detection and controlsystem of the invention are trained to continuously analyze varioustypes of image data to recognize, quantize and characterize fire imagesthroughout the fire fighting effort. Training the network involvesproviding known inputs to the network resulting in desired outputresponses. The weights are automatically adjusted, based on error signalmeasurements, until the desired outputs are generated. Various learningalgorithms may be applied. Adaptive operation is also possible withonline adjustment of network weights to meet imaging requirements.

The neural network configuration of the image analysis computers ofFIGS. 5 and 6 is preferably implemented in a highly parallel imageprocessing structure, enabling rapid image analysis and recognitionnecessary for optimizing fire detection and decision making real timeand automatic fire fighting decision. Very Large Scale Integrated (VLSI)circuit implementations of the neural processing elements provides arelatively low cost but highly reliable system important to a warningand automatic control system of the type herein disclosed. Inparticular, loss of any one processing element does not necessarilyresult in a processing system failure.

Each of the programming techniques is well known to those of ordinaryskill in the art, as discussed in the various references incorporated byreference above, and accordingly, are not repeated in detail here. Otherprocessing implementations can be substituted. For example, in thealternate embodiment shown in FIG. 9, the neural computing network isimplemented with multiple virtual processing elements 180 coupled to animage processor 170. Image data is presented to the image processor 170over data bus 175 is routed to selected virtual processing elements 180,which implement the neural network computing functions. The virtualprocessing elements 180 may comprise pipe-lined processors to increasethe overall speed and computational efficiency of the system.

In its preferred embodiment, the expert system control logic for controlprocessor 208 of FIG. 6, employs fuzzy logic algorithmic structures togenerate the command-control and warning signals. Fuzzy logic isparticularly well suited to implement and help solve a complex problem,and to handle and analyze the multiplicity of image signals andenvironmental parameters generated, each of which may be defined by arange of values in different combinations which require differentcomputing and fire fighting responses.

Using the previously described satellite video and infrared scanning anddetecting methods, together with accurate location and trackinginformation using satellite positioning systems (e.g., GPS or GLONASS),the control processor 208 of FIG. 6 accurately locates and generatecodes defining the global locations of persons and/or valuableproperties in the sectors depicted in FIG. 3. The control processor 208,image processor 205 and database computer 209 analyze the contents ofimages of the monitored areas to fully characterize any fires. Forexample, the computer systems and image analysis methods determine thecontour and the distance(s) between the edges of the fire, the locationsof particular persons and/or valuable property in the path of the fire,the distance(s) from a select portion of an edge of a particular fire(such as one or more fires F3 and/or F2 shown in FIG. 4) to thelocations of particular persons, properties or fire fighting resources30,40. Such distances define parameters which, when compared with thedetermined rate of the fire spread in that direction, determines thedegree of danger to a particular location at any particular instant intime as the fire progresses.

In addition to detecting such distance(s), the computing systems, imageanalysis and fuzzy logic methods of this invention characterize othercritical fire fighting factors to continually monitor and identifyexisting degrees of danger in various of the monitored geographicsectors. For example, the information relating to extent, location andcombustibility of materials between the fire and the known location ofpersons, valuable property, natural resources, and fire fightingresources is used to further prioritize fire fighting decisions. Forexample, very dry conditions may be known to exist in a dense forestarea, and further, that there is a high degree of combustible materialpresent on the forest floor. Those conditions represent a very hazardoussituation in the presence of fire. On the other hand, wide open areaswith little or dispersed vegetation, or very wet conditions, represent acomparatively less dangerous situation. Other factors such as thepresence of highways, rivers, lakes, intervening mountains, firebreaksor other natural or man-made barriers in the path of a fire furtherreduce the risk or danger factor in those areas. Thus, in a preferredembodiment, a combustion factor is defined and stored in the database210 for each of the monitored sectors A11, A12, depicted in FIG. 3.

Another critical fire control factor is weather condition, such as thepresence of a high wind, or in contrast, a driving rain. Thus, anotherinput to the control computers is continuously updated weatherconditions in the monitored areas. For example, weather conditionsindicating the presence of high winds passing from a fire to an area ofhigh concern (either due to population, natural resources, propertyvalues, wildlife, etc.) increases the risk or priority to that area. Onthe other hand, no wind, low wind or driving rain considerably reducesrisk in that area.

Accordingly, one of the purposes of the remote fire control centers21-26, satellites 10, surveillance craft 11, and the fire fightingresources 30,40, is to continually update the computers at fire controlheadquarters 20 with current weather information, including specificallywind and rain conditions. In addition, such weather conditions can becontinually downloaded from existing space or earth bound weatherstations. It is preferred that the remote fire control centers 21-26,and various of the fire fighting resources 30,40, contain sensors thatcontinually measure important weather conditions such as wind velocityand direction and communicate that data continually or on-demand to thefire control headquarters 20. The weather information received at firecontrol headquarters 20 is analyzed and formatted for input to the fuzzylogic expert systems to optimize fire fighting decisions, and fordisplay on terminals 212. Extremely high or increasing winds will notonly impact fire fighting decisions, but may also result in thegeneration and communication of audible alarms and speech synthesizedwarnings, as discussed previously.

Determining the combined impact of critical fire fighting variables inan organized, accurate and real-time basis to assess or predict relativedanger and priority zones requires a structured approach based on expertsystem knowledge and past experience with respect to risk assessment.Fuzzy logic electronic circuitry and software is a particularlyattractive method for implementing such an expert system to determineand quantify the relative degree of danger for sectors of a monitoredregion, such as that depicted in FIGS. 3 and 4.

In a preferred embodiment, a danger index is derived for selected (orall) sectors of the monitored region, using fuzzy logic inference rulesto evaluate critical fire fighting control factors, such as: thedistance or distances between the fire and one or more locations ofconcern, the combustion factor between the fire and each location, thevelocity of the wind in the direction from the fire to each location ofconcern, property values in the selected area, etc. Each of theseparameters is computer analyzed and evaluated on a real-time basis andits quantified codes used in the fuzzy logic expert system to assist inoptimizing fire fighting activities.

Additional factors critical to optimizing fire fighting decisionsinclude the precise location and type of all available fire fightingresources 30,40. The fuzzy logic expert systems of the present inventionevaluate the derived relative danger factors and the known location ofall available fire fighting resources 30,40 to prioritize and automatedispatching decisions. As discussed above, in the preferred form, thefire fighting resources 30,40 are continually monitored to determinetheir precise geographic location using GPS satellite positioning andassociated data communication methods. If data defining images of thefire or weather are transmitted from remote control centers 21-26,non-orbiting surveillance craft 11, or other fire fighting resources30,40, such data is also “tagged” with GPS-based identifiers. In thatmanner, all relevant information is stored in the data base 210 and usedby the control processor 208 in a manner that is keyed to precisegeographic data. Thus, data defining relevant fire control factors aretransmitted to the fire control headquarters 120 on a real-time basisover communication links 70. Such data communication is managed by thecommunication computer or processor 215, and the database computer 209and associated storage unit 210, as depicted in FIG. 6. The dataacquired and stored at the fire control headquarters is thereafter usedin comprehensive fuzzy logic analyses to optimize fire controldecisions.

FIGS. 10A-10D illustrate exemplary fuzzy logic membershipclassifications for four of the fire control factors discussed above.The classifications and memberships disclosed in FIGS. 10A through 10Dare to be considered as examples only, and can be expanded, contractedor varied to accommodate additional, fewer or different fire controlfactors.

FIG. 10A illustrates possible membership grades for distance(s) betweenthe location(s) of concern and the spreading fire or fires. Usingstandard fuzzy logic membership classification methods, overlappingmembership categories are defined for such distances as very close,close, medium, far and very far. Trapezoidal membership classificationis used with the sum of the overlapping memberships always equal to one.The actual distance(s) may be measured in feet, miles, meters orkilometers, for example, as appropriate and most convenient in aparticular situation. Fire fighting experts can apply their extensiveknowledge from past fire fighting experiences to classify the distancesin the appropriate membership grades.

The combustion factor variable is similarly defined as shown in FIG.10B, as an input to the disclosed fuzzy logic control system. Fivemembership classifications are defined corresponding to very low, low,normal, high and very high combustion situations in each of the areasdepicted in FIGS. 3 and 4. As noted above, the combustion factors willdepend upon features of the natural environment, whether thatenvironment is dry or wet, and the presence, location and extent ofnatural or man-made fire breaks that are located in particular areas.The appropriate combustion factor between the fire and a particularlocation in the area being monitored is considered as the weightedaverage of a number of such combustion factors for the intervening areasbetween the fire and the location of concern. As an alternative, thehighest combustion factor in the path of a fire is used to determine thedegree of fire danger at a particular location. Other averagingapproaches where multiple areas exist between the fire and the locationof concern can be used, and will be apparent to experienced systemprogrammers using the disclosed fuzzy logic methods.

FIG. 10C illustrates exemplary membership grades for wind factor as aninput variable to the disclosed fuzzy logic inference rules. As shown,the membership grade varies between 0 and 1, with overlapping membershipcategories based on the assessment of danger represented by differentwind conditions. The wind factor evaluated is the velocity of the windin the direction from the fire to the location of concern. The windfactor may be evaluated from actual wind measurements and from the rateof spread of the fire as determined by video and/or infrared scanning.The appropriate wind velocity vector can be calculated using well knownvector algebra and trigonometric techniques based on the actual winddirection and velocity and the direction from the fire to the locationof concern in the monitored area. Four categories of wind velocity inthe direction from the fire to the location of concern are shown in FIG.10C: no wind, low wind, moderate wind and high wind. Once again, firefighting experts are able to define appropriate ranges for the variouswind velocity classifications shown in FIG. 10c.

FIG. 10D illustrates fuzzy logic membership grades for the output of thefuzzy logic inference system and method discussed above. The outputvariable is a computed fire danger index applicable for analyzingparticular sectors or locations in the monitored region illustrated inFIGS. 3 and 4. Five dassifications of danger are illustrated in FIG.10D: very low, low, normal, high and very high. The danger indexclassifications overlap in accordance with standard fuzzy logicprinciples. In the example shown in FIG. 10, the danger index for aparticular sector or location in FIGS. 3 and 4 is calculated using fuzzylogic inference rules based on the selected input variables of distance,combustion factor(s) and wind factor(s) as illustrated in FIGS. 10A, 10Band 10C. The result is a numerical danger index defined in accordancewith the overlapping fuzzy logic membership classifications of FIG. 10D.

FIG. 11 presents exemplary fuzzy logic inference rules used by an expertsystem to evaluate the three input variables defined in FIGS. 10A, 10Band 10C, and employed to produce the appropriate danger index outputclassification according to FIG. 10D. Four illustrative tables are shownin FIG. 11, corresponding to wind classifications extending between andin the directions from the fire to the area of concern and defined asnone, low, moderate and high corresponding to the membershipclassifications of FIG. 10C. The combustion factor and distancevariables are represented by the rows and columns of each respectivematrix of FIG. 11. Reading across the column headings, distancescorresponding to very dose (VC), close (C), medium (M), far (F), andvery far (VF) are indicated. Similarly, combustion factors of very low(VL), low(L), normal (N), high (H) and very high (VH) are indicated inthe respective rows. The values in each of the respective matrices thencorresponds to the classifications of the danger index as represented inFIG. 10D. That is, the danger indices are indicted as being very low(VL), low (L), medium (M), high (H) and very high (VH). For example,reading from the danger index fuzzy logic inference rule tablecorresponding to low wind, selected inference rules are indicated asfollows:

IF: (i) WIND=LOW, and (ii) DISTANCE=FAR, and (iii) COMBUSTIONFACTOR=HIGH, then DANGER INDEX=MEDIUM.

IF: (i) WIND=LOW, and (ii) DISTANCE=CLOSE, and (iii) COMBUSTIONFACTOR=HIGH, then DANGER INDEX=VERY HIGH.

Similarly, representative danger indices read from the matrixcorresponding to high wind are as follows:

IF: (i) WIND=HIGH, and (ii) DISTANCE=VERY CLOSE, and (iii) COMBUSTIONFACTOR=NORMAL, then DANGER INDEX=VERY HIGH.

IF: (i) WIND=HIGH, and (ii) DISTANCE=MEDIUM, and (iii) COMBUSTIONFACTOR=NORMAL, then DANGER INDEX=HIGH.

IF: (i) WIND=HIGH, and (ii) AND DISTANCE=MEDIUM, and (iii) COMBUSTIONFACTOR=VERY HIGH, then DANGER INDEX=VERY HIGH.

As can be seen from the danger index fuzzy logic inference rulesdepicted in FIG. 11, the expert system of the present invention reducesa potentially complex set of input information defining the fire and theenvironmental parameters to a manageable fuzzy logic calculationresulting in a danger index associated with each (or selected) of thesectors of the monitored area.

Depending on the values of particular input parameters, and theoverlapping regions of those parameters, as defined in the membershipgrades of FIGS. 10A, 10B and 10C, more than one of the fuzzy logicinference rules of FIG. 11 may be triggered by a given set ofcircumstances. In such cases, the most appropriate danger index can becalculated using well known fuzzy logic calculation procedures fordefuzzification, such as the centroid method. This method permitsreal-time evaluation of the danger index for each of the areas of amonitored region such as shown in FIGS. 3 and 4, or for particularlocations within regions depending upon the requirements and firefighting situation. Combining the calculation of a specific danger indexfor each area with real-time display of the location and type of firefighting resources 30,40, and the use of advanced communication of suchinformation to the entire fire fighting team, allows fire fightingcommanders to make proper decisions based on a large amount of complexbut current data.

FIG. 11A further illustrates additional fuzzy logic inference rulesapplicable to the fire fighting control systems and methods of thepresent invention. The additional inference rules of FIG. 11A addconsideration of another fire fighting factor—the measured rate ofspread of the fire. As discussed in greater detail above, the rate ofspread of the fire may be determined from data received from the videoand/or infrared scanning information derived from the satellite 10 orairborne 11 scanners of FIGS. 1 and 1A, from the remote fire controlcenters 21-26, or from the fire fighting resources 30,40. Three spreadrates are indicated in FIG. 11A: low, medium and high. Correspondingtrapezoidal membership functions are defined to incorporate theadditional information into the total fuzzy logic calculation of dangerindices. The fuzzy logic inference rule tables of FIGS. 11 and 11A arereferred to as Fuzzy Associative Memories (FAM's) that are rapidlyaccessed based on parameter values for real time calculations.Additional FAM's can be developed by the skilled artisan to morecompletely consider further fire control factors, such as the knownlocation of dispatched fire fighting resources.

Thus, the above-described danger index calculation method is used toquickly calculate, in real-time, a danger index for each of the areasA₁₁, A₁₂, A₁₃ . . . defined in FIG. 3. The resulting data is organizedin a danger index matrix, D, as illustrated in FIG. 12. In thepresentation of FIG. 12, the rows of the danger index matrix Dcorrespond to the eight sectors represented in FIG. 3, while the columnscorrespond to the five zones in the monitored area. With thisdefinition, the danger index matrix D is a 8×5 matrix. The dangerindices D₁₁, D₁₂ . . . correspond to the calculated danger in thecorresponding sector of the monitored region. If desired, the dangerindex matrix can be further sub-divided to reflect particular locationswithin each sector or area.

The danger index matrix D is used to help fire fighters optimize firefighting decisions. In more advanced forms, the danger index matrix usesadditional information concerning the relative importance or value ofdifferent areas within the monitored region. Such value assignments aremade based on the presence of persons and/or the value of property orother natural resources (e.g., wildlife preserves, loggable timber,etc.). Such factors are used to assign a value index to each of theareas in the region of FIG. 3. The relative value indices are in turnorganized in a value matrix, V, such as illustrated in FIG. 13. The rowsof the matrix V correspond to the 5 circular zones of FIG. 3, and thecolumns in turn correspond to the 8 sectors. Thus, the value variableV23 represents the value assigned to the area defined by zone 2 andsector 3 of FIG. 3.

The expert system uses the value matrix entries of FIG. 13 and thedanger index values of FIG. 12 to determine more specific priorities forfighting fires in particular areas of the monitored region. For example,a priority indication is obtained by multiplying the danger index foreach area by the value ascribed to that area. Areas that have alreadyburned or where the fire is under control, may be indicated with a lowvalue of danger index which assures a low value of priority in terms ofdispatching of fire fighting resources. Ranking the results for eacharea in descending order defines a set of priorities for fighting thefire. The control processor 208 recommends or initiates the dispatchingof appropriate fire fighting resources 30,40 to the higher priorityareas. Again, by continually tracking the type and location of allavailable and committed fire fighting resources 30,40, the controlprocessor 208 uses expert system and fuzzy logic reasoning to optimizeallocation of resources to all high priority areas. As a result, thebest available fire fighting resources 30,40 are properly dispatched ona real-time basis to the areas in immediate danger, and such resourcesare not diverted in the areas of lowest priority.

It still another refinement of the preferred embodiment, it is desirableto prioritize the eight sectors shown in FIGS. 3 and 4 by computing forthe sector of interest the “inner” or “dot product” of the respectiverow and column vectors of the value matrix V of FIG. 13 and danger indexmatrix D of FIG. 12. For example, a priority value for sector 1 as awhole is obtained as the sum of the products of the individual matrixelements from row 1 of the value matrix V and column 1 of the dangermatrix D. Repeating this calculation for each of the various sectorswill result in a sector priority vector as indicated in FIG. 14. Usingsuch a vector, fire fighting resources are dispatched to the highestpriority sector.

FIG. 15 illustrates in graphic form an example danger index calculationbased on the above-described fuzzy logic reasoning. The calculation ofFIG. 15 corresponds to a computation of the danger index for aparticular area within a monitored region, for example, areas Aij ofFIG. 3. In the case illustrated in FIG. 15, particular values exist forthe wind index, the combustion index and the distance index, each ofwhich serves as an input to the fuzzy logic computation used todetermine a resultant danger index for the area Aij. For the values ofthe wind, combustion, and distance variables that exist, eight of thefuzzy logic inference rules of FIG. 11 are shown. Two of these areillustrated in FIG. 15. The first corresponds to the “Wind=Low” table ofFIG. 11, and is stated as rule 1 in FIG. 15 as:

IF: (i) WIND=LOW, (ii) COMBUSTION=NORMAL, and (iii) DISTANCE=VERY CLOSE,then DANGER=VERY HIGH.

The second corresponds to the “Wind=Moderate” table of FIG. 11, and isstated as follows:

IF: (i) WIND=MODERATE, (ii) COMBUSTION=LOW, and (iii) DISTANCE=CLOSE,then DANGER=VERY HIGH.

As shown in the graphic analysis of FIG. 15, each of the input variablesis in the “fuzzy area” of its definition between respective membershipclassifications. The particular wind value, for example, is somewherebetween low and moderate, while the combustion index is between low andnormal, and the distance is between dose and very dose. Using fuzzylogic calculation procedures, the corresponding value of the dangerindex for each combination of the indicated values is computed. Two ofthe eight calculations are actually shown in FIG. 15. Both the high andvery high danger index membership classifications are involved in thetwo example computations. As also indicated in FIG. 15, it is necessaryto combine the results from the very high and high danger indexclassifications in a “defuzzification” process to compute an appropriateresultant value for the final danger index for the given conditions. The“defuzzification” is accomplished by overlaying the respective areasfrom the computations to identify the intersection of the respectivemembership classification variables. Using such standard fuzzy logicprocedures, defuzzification is accomplished by computing the centroid ofthe resulting areas as indicated in FIG. 15. In fact, eight such resultswould be included in the centroid calculation. Of course, otherdefuzzification procedures may be used depending on the particularalgorithm implemented in the methods herein taught.

The procedure discussed in connection with FIG. 15, is carried out foreach of the areas Aij in FIG. 3, corresponding to the monitored region.The resultant calculations result in the danger index matrix illustratedin FIG. 12. The danger index values are used in computing the firefighting priority vectors in accordance with the methods describedabove.

The use of a rectangular coordinate system with correspondingrectangular sub-areas, as illustrated in FIG. 16, is particularly usefulin assigning relative priority values to individual sub-areas. Suchconversion is accomplished by overlaying the circular evaluation patternof FIGS. 3 and 4 over the rectangular coordinate map of FIG. 16, asillustrated in FIG. 17. A corresponding rectangular area priority matrixis shown in FIG. 18. Superimposing the circular evaluation pattern ofFIGS. 3 and 4 over the rectangular coordinate map of FIG. 16 permitsevaluation of relative values of the circular sub-area sectors byintegrating and computing the fraction of each rectangular sub-areacontained within the corresponding circular sub-areas.

For example, assuming a uniform distribution of value across therectangular areas permits evaluation of a value corresponding to thecircular sub-sector areas. Thus, a database premised on a rectangularcoordinate system containing sub-area values can be converted to thecircular areas described earlier. Also, in another embodiment, it mayprove desirable to use the rectangular coordinate system without thecircular sector analysis. Those skilled in the art will be able toeasily adapt the above-described calculation procedures for circularareas to a rectangular coordinate system with similar fuzzy logicevaluations and prioritization.

Another useful method for optimizing fire fighting decisions is toconsider priorities in each area and in surrounding areas. This may beaccomplished, for example, by summing priorities immediately adjacent toeach individual area with the priority for each area to result in anadjacent node priority matrix as shown in FIG. 19. This adjacencyapproach may indicate, for example, that it is important to fight a firein a particular area because of the fire danger to adjacent areas.

FIG. 20 is a preferred flow diagram for the fire control processcomputation methods described above. The overall process begins in block220, which selects a new area to be monitored by the fire fightingsystem. Block 221 corresponds to the actual imaging or scanning of theselected area using any of many available imaging systems, such as thesurveillance satellite 10 or craft 11, shown in FIG. 1 or 1A. If no fireis so detected, another area for scanning is automatically selected asindicated in FIG. 20. The image data may be analyzed in the surveillancesource 10,11, or transmitted to the fire control headquarters foranalysis, as shown in block 223. In block 224 the image data received atthe fire fighting control headquarters 20 from the surveillance source10,11 is further analyzed by computer to fully characterize allimportant fire fighting factors, such as the fire's precise location,rate of spread, intensity, etc. At the same time, terrain and value datais retrieved from the database 210, as referenced in block 225. Amongother things, the terrain data describes and characterizes thecombustion factor index, as described in connection with FIG. 10, andvalues for the area value matrix, as described in connection with FIG.12. As noted above, the combustion factor and value indices may bechanged from time to time depending upon environmental changes and fromsurveys of areas to be monitored. Wind data is also obtained from thedatabase retrieval system as referenced in block 226. The wind data, ofcourse, is updated on a real-time basis from remote control centers21-26 and the fire fighting resources 30,40, as discussed above.

Based on the received image data and the selected fire control factors,the danger index matrix is calculated as indicated in block 227 of FIG.20. Based on the results of the danger index calculation, the sectorpriority vectors are computed in block 228. The sector priority vectorsdefine the highest priority sectors for the dispatching of fire fightersand fire fighting equipment as indicated in block 229 of FIG. 20.Although not shown in FIG. 20, the control processor 208 also tracks andmonitors the precise location of all available fire fighting resources30,40, and automatically optimizes fire fighting decisions by evaluatingthe priority values in view of the known location of the fire fightingresources.

Using the fire fighting systems and methods disclosed above, anautomated online expert system evaluation is implemented to assist firefighters in their decision making process on a real-time basis, and tooptimize fire fighting efforts. Real-time update of prioritizationinformation based on the status of the fire and environmental conditionsalong with relative values of various areas in the path of the firegreatly assists fire fighters and decision makers during criticalperiods of fire fighting activities. Of course, ultimate dispatchingdecisions are made by fire fighting personnel who are able to evaluatemultiple parameters and dangerous situations on a real-time basis andmake appropriate fire fighting decision. The system herein abovedescribed provides yet another source of information to assist in makingthese decisions.

The inventions set forth above are subject to many modifications andchanges without departing from the spirit, scope or essentialcharacteristics thereof. Thus, the embodiments explained above should beconsidered in all respects as being illustrative rather than restrictiveof the scope of the inventions, as defined in the appended daims.

What is claimed is:
 1. A fire fighting control system comprising: a. asurveillance craft including: (i) an imaging device, (ii) a controllercoupled to the imaging device, and (iii) a communication circuit coupledto the controller; b. a fire control headquarters having: (i) a computercontrol system including a display system, a control program memory thatstores a control program, and a database memory that stores datacharacterizing predefined fire control parameters for selectedgeographic regions of the earth, (ii) a neural network processorprogrammed to analyze image signals and detect and characterize fires inthe captured image, (iii) a first communication link between thecomputer control system and the communication circuit of thesurveillance craft, and (iv) a second communication link between thecomputer control system and at least one fire fighting resource; and c.wherein the control program includes: (i) image acquisition commandsthat are executed by the computer control system to direct thesurveillance craft to obtain and transmit to the computer control systemimage data defining images of selected geographic regions of the earth,(ii) image analysis commands that are executed by the command controlsystem and neural network processor to analyze the communicated imagedata to determine whether a fire exists in a selected geographic region,and if a fire is determined to exist, to characterize selected featuresof the fire, (iii) database acquisition commands that are executed bythe computer control system to locate and acquire from the database datadefining fire control parameters characterizing the selected geographicregion in which a fire exists; (iv) expert system analysis commands thatare executed by the computer control system to analyze the data definingthe selected features of the fire and the fire control parameterscharacterizing the selected geographic region in which the fire exists,and to render recommended fire fighting decisions, and (v) communicationcommands executed by the computer control system to control the displayterminal to indicate recommended fire fighting decisions, and tocommunicate selected of the decisions to at least one fire fightingresource.
 2. The fire control system of claim 1 further comprising ageographic positioning system that tracks the location of fire fightingresources and communicates to the computer control system data definingthe tracked location.
 3. The fire control system of claim 2 wherein theexpert system analysis commands direct the computer control system toanalyze the data defining the location of fire fighting resources inconnection with rendering recommended fire fighting decisions.
 4. Thefire control system of claim 1 wherein the data characterizingpredefined fire control parameters stored in the database includes datadefining the terrain of the selected geographic regions.
 5. The firecontrol system of claim 1 wherein the data characterizing predefinedfire control parameters stored in the database includes data definingthe value of property in the selected geographic regions.
 6. The firecontrol system of claim 1 wherein the data characterizing predefinedfire control parameters stored in the database includes data definingthe population in the selected geographic regions.
 7. The fire controlsystem of claim 1 wherein the data characterizing predefined firecontrol parameters stored in the database include data defining: (i) theterrain of the selected for selected geographic regions, (ii) the valueof property in the selected geographic regions, and (iii) population inthe selected geographic regions.
 8. The fire control system of claim 1wherein the expert system computer is controlled to execute selected ofthe expert system analysis commands and to apply predefined rules toanalyze the data defining selected features of the fire and the firecontrol parameters to assign a danger factor to portions of the selectedgeographic region in which the fire is located.
 9. The fire controlsystem of claim 1 wherein the surveillance craft is a satellite orbitingthe earth.
 10. The fire control system of claim 9 wherein thesurveillance craft is a remote controlled aircraft.
 11. The fire controlsystem of claim 9 wherein the surveillance craft is a manned aircraft.12. The fire control system of claim 1 further comprising a weathergathering station coupled to the computer control system by acommunication link and controlled to monitor and generate datacharacterizing selected weather conditions in its vicinity.
 13. The firecontrol system of claim 12 wherein the control program includes weathercontrol commands that are executed by the computer control system todirect the weather gathering station to communicate to the computercontrol system the data defining weather conditions.
 14. The firecontrol system of claim 13 wherein the expert system analysis commandsdirect the computer control system to analyze the data defining theweather in the vicinity of the weather gathering station in connectionwith rendering recommended fire fighting decisions.
 15. The fire controlsystem of claim 1 wherein the computer control system includes a speechsynthesizing computer coupled to the expert system computer.
 16. Thefire control system of claim 15 wherein the expert system communicatesto the speech synthesizing computer data defining recommended firefighting decisions.
 17. The fire control system of claim 16 furthercomprising speakers coupled to the speech synthesizing computer, andwherein the speech synthesizing computer operates on the data definingrecommended fire fighting decisions and generates speech data definingaudible commands that are issued over the speakers.
 18. The fire controlsystem of claim 17 wherein the speech synthesizing computer is coupledto the communication link that exists between the computer controlsystem and the fire fighting resources, and wherein the data definingthe audible commands generated by the speech synthesizing computer arecommunicated over the communication link to selected fire fightingresources.
 19. The fire control system of claim 1 wherein the imageanalysis commands include commands that are executed by the commandcontrol system to characterize the location of a fire.
 20. The firecontrol system of claim 1 wherein the image analysis commands includecommands that are executed by the command control system to characterizethe direction of movement of a fire.
 21. The fire control system ofclaim 1 wherein the image analysis commands include commands that areexecuted by the command control system to characterize the intensity ofa fire.
 22. The fire control system of claim 1 wherein the imageanalysis commands include commands that are executed by the commandcontrol system to characterize the location and direction of movement ofa fire.
 23. The fire control system of claim 22 wherein the expertsystem analysis commands direct the computer control system to analyzethe data defining the location and direction of movement of a fire inconnection with rendering recommended fire fighting decisions.